Capability analyses of mdtmFTP

A decade ago it was theorized that if enough parallel processing power was thrown at an I/O problem, the bottleneck would shift towards the NIC’s link speed [1]. However, today NUMA architecture changes this paradigm, it also matters on which cores these threads are located and how they are managed [2]. Transport protocols are also critical in high latency networks, TCP’s congestion control algorithm is window based and every time a congestion event is detected, the window size is reduced to half. This congestion control algorithm has significant limitations on HBDP (High Bandwidth Delay Product) network environments [3]. This research project report delves into the capabilities of mdtmFTP (Multicore-Aware Data Transfer Middleware File Transfer Protocol) [4] and compares it to Globus GridFTP [5] using transatlantic data transfer experiments on a 100 Gbit/s light path between Amsterdam and Chicago with a RTT (Round Trip Time) of 95 ms. mdtmFTP aims to accelerate data movement toolkits on multicore systems by using an application level resource scheduler which is aware of the underlying hardware topology. Globus GridFTP will be configured with both TCP and UDT [6] to evaluate UDT’s effectiveness on HBDP network environments. UDT is an application-level data transport protocol that uses UDP to transfer bulk data, while implementing its own reliability and congestion control mechanisms and can be enabled as a Globus XIO driver in Globus GridFTP [1]. A variety of experiments will conclude which tool optimally utilizes the 100 Gbit/s transatlantic light path.